Decision Making without Likelihood Judgment 1 On Decision Making without Likelihood Judgment

Subjective expected utility, prospect theory and most other formal models of decision making under uncertainty are probabilistic: they assume that in making choices people judge the likelihood of relevant uncertainties. Clearly, in many situations people do indeed judge likelihood. However, we present experiments suggesting that there are also many situations in which people do not judge likelihood and instead base their decisions on ad-hoc rules. Thus, we argue that real-world situations are of two types. In situations eliciting a probabilistic mindset, people rely on judgments of likelihood. In situations eliciting a non-probabilistic mindset, people rely on ad-hoc rules. We discuss factors that may influence the tendency to engage in probabilistic versus non-probabilistic mindsets and how extant probabilistic models may be complemented by non-probabilistic models. Decision Making without Likelihood Judgment 3 Most theories of decision making under uncertainty assume that a decision maker choosing among a set of options (e.g., travel to China vs. do not do so) assesses the desirability of each option under each potential course of events (e.g., the trip would be fun if SARS is contained, unpleasant if it resurges), judges the likelihood of each course of events, and combines assessments of desirability and beliefs about likelihood to arrive at a choice (e.g., take the trip if containment is sufficiently likely, don’t otherwise). Critically, then, most theories are probabilistic: they take it as given that in making decisions, people judge the likelihood of relevant events. The probabilistic approach underlies normative accounts such as subjective expected utility as well as descriptive accounts such as prospect theory. That both standard normative models and preeminent descriptive models are probabilistic has not gone unnoticed. As Kahneman remarked: “[prospect] theory ... did not challenge the ... analysis of choices in terms of beliefs [about likelihood] and desires that underlies utility theory (Kahneman, 2000).” It is clear that in many situations people do indeed make decisions by judging the likelihood of relevant events. In this article, however, we pursue a challenge to the probabilistic approach, by presenting experiments suggesting that in many situations people do not judge the likelihood of relevant events and instead base their decisions on ad-hoc rules or rationales. In essence, we argue that real-world situations are of two types. In situations eliciting probabilistic mindsets, people rely on judgments of likelihood. In situations eliciting non-probabilistic mindsets, people do not judge likelihood and instead appeal to adhoc rules or rationales. Probabilistic models such as expected utility and prospect theory will fit best when people hold probabilistic rather than non-probabilistic mindsets. To formally distinguish probabilistic and non-probabilistic mindsets, consider situations defined by the triples (EL, x, f) and (EH, x, g). Here, EL and EH are possible events, the former of lower perceived likelihood than the latter. The decision-maker receives attractive outcome, x, in the former situation only if Decision Making without Likelihood Judgment 4 EL obtains and in the latter only if EH obtains. All other features of the two situations are captured by f and g. Probabilistic and non-probabilistic mindsets focus on different subsets of these variables. Suppose an individual is choosing which situation to enter. Someone with a probabilistic mindset will judge the likelihood of EL and EH (denote these judgments p(EL) and p(EH)) and will choose the latter situation, precisely because it provides a greater probability of getting x (i.e., p(EH)> p(EL)). In contrast, someone with a non-probabilistic mindset will rely on a rule that depends on f and g (and perhaps x as well) but that neglects EL and EH. If f is more consistent with the rule than is g, the individual will choose the former situation over the latter. Our experiments expose such patterns of opposing preferences. The remainder of the paper is organized as follows. We first review prior research suggesting the existence of non-probabilistic mindsets. We then turn to our experiments. Throughout, we discuss factors that may influence the tendency to engage in probabilistic versus non-probabilistic mindsets. Research Suggesting Non-Probabilistic Mindsets Phenomena that may reflect non-probabilistic mindsets have been documented in several areas of psychology and related fields. In an important field study, Shapira (1997) observed that “rather than formulating probability estimates, executives create potential scenarios based ... on ... arbitrary factors inherent in their own situations” and that reactions to such scenarios guide choices. In our terminology, Shapira observed that executives often hold non-probabilistic mindsets; they base their decisions on rules or rationales reflecting “arbitrary factors” such as f or g. Pennington and Hastie (1988, 1993) uncovered similar reliance on scenario construction in juror behavior. March (1994) describes a potential second source of non-probabilistic behavior, considerations of appropriateness: Decision Making without Likelihood Judgment 5 “When individuals ... follow rules or procedures that they see as appropriate ... Neither preferences ... nor expectations [of likelihood] ... enter directly into the calculus ... Decision makers ... ask ... [a] what kind of situation is this? ... [b] what does a person such as I ... do in a situation such as this? (pp. 57-58)” Consistent with March’s question [a] Klein and colleagues assert that expert decision making often proceeds by the recognition of a situation as of a kind for which there is a prepared course of action (Klein, Orsanue, & Calderwood, 1993; Zsambok & Klein, 1997). Consistent with March’s question [b], the work of Prelec and Herrnstein (1991) and Baron and Spranca (1997) can be interpreted as suggesting that reliance on moral principles sometimes crowds out likelihood judgment. Loewenstein et al (2001) and Slovic et al (2002; Slovic, 1987; Peters & Slovic, 2000) study what may be a third source of non-probabilistic behavior: affect. They find that emotional reactions to uncertainty are sometimes at odds with likelihood judgments, and, critically, that when such divergence occurs, emotional reactions often drive decisions. Erev and Roth examine behavior when people repeatedly encounter similar decisions. They observe a potential fourth form of non-probabilistic behavior. Choices in repeated decisions often accord with reinforcement-learning principles that can be inconsistent with probabilistic considerations (Erev & Roth, 1998; Roth & Erev, 1995; Barron & Erev, 2003; Camerer & Ho, 1999; Ratner & Herbst, 2003). Finally, studies by Busemeyer and Townsend (1993; Diederich & Busemeyer, 1999) can be interpreted as indicating that shifts in people’s attention from one potential consequence to another, rather than likelihood judgments, determine choices (see also March & Shapira, 1992). The diverse phenomena just reviewed reflect numerous psychological mechanisms. One contribution of our framework lies in indicating that all these phenomena share an essential property: they are influenced by “arbitrary features” of situations. In a situation defined by (E,x,f), probabilistic mindsets draw on the Decision Making without Likelihood Judgment 6 likelihood p(E) and the desirability of x. But non-probabilistic mindsets may neglect likelihood and depend on how the outcome x and features f influence (i) the scenario constructed, (ii) assessments of appropriateness, (iii) emotional reactions, (iv) the salience of similar past decisions, or (v) patterns of attention. Two streams of work presage the experiments we report. Hogarth and Kunreuther (1995) found that participants provided with probability information incorporated this factor into their choices, but that participants not provided with probability information relied on “arguments [that] processed the particular characteristics of each choice option.” Rettinger and Hastie (2001), Goldstein and Weber (1995), Beach (1990), and MacCrimmon and Wehrung (1986) observed that certain contexts facilitate probabilistic approaches, whereas other contexts facilitate non-probabilistic approaches. Our experiments build on these investigations by contrasting probabilistic and non-probabilistic mindsets in settings where information and context are held constant across participants; in our studies priming manipulations spur participants’ focus towards either probabilistic or non-probabilistic variables. Experiment 1: Sandwiches Method University of Chicago students (n=396) were asked to imagine that they could participate in a frequent buyer program that would reward them for purchasing sandwiches at local shops. Participants were randomly assigned to one of eight conditions in a 2 (purchase requirement: low vs. high) x 2 (reward magnitude: small vs. large) x 2 (probability prompting: prompted vs. unprompted) between-subjects design. All participants were told that joining the program required paying a $2 membership fee. Participants’ decisions of whether or not to join formed our main dependent measure. Decision Making without Likelihood Judgment 7 Participants were told that they would receive a reward after purchasing either ten sandwiches (low requirement) or twenty sandwiches (high requirement) and that their reward would be either one free sandwich (small magnitude) or two free sandwiches (large magnitude). Probabilistic and non-probabilistic mindsets may engender different “construals” of purchase requirements (e.g., Ross & Nisbett, 1991). A probabilistic person judges his or her likelihood of making the required number of purchases. The fewer sandwiches required, the better the chances of meeting the requirement. Thus, a probabilistic person may construe a high purchase requirement as engendering a “low probability” of reward and a low purchase requirement as engendering a “high probability” of reward. A non-probabilistic person neglects to judge his or her likelihood of meeting the purchase requirement and thus will not construe different requirements as “low probability” or “high probability” events. We suggest that a non-probabilistic person may interpret the situation as a potential trade of effort for reward and may decide whether to join the program using a reciprocity rule: “when the effort required of me is significant, I deserve a significant reward” (Kivetz, 2003 discusses effects of effort on reward preferences). Thus, a non-probabilistic person may construe a low purchase requirement as necessitating “low effort” and a high purchase requirement as necessitating “high effort.” Reflecting differing construals, probabilistic and non-probabilistic mindsets yield opposite predictions about how the tendency to join will be influenced by the interaction of purchase requirement and reward magnitude. Given a high purchase requirement, probabilistic people construe a “low probability” of reward and nonprobabilistic people a “high effort” requirement. In this case, probabilistic people may not distinguish Decision Making without Likelihood Judgment 8 between different reward magnitudes; they may find reward magnitude immaterial, because the reward is unlikely to be received, anyway. On the other hand, non-probabilistic people may distinguish between reward magnitudes; a small reward might not reciprocate high effort but a large reward might. In sum, given a high purchase requirement, the difference in reward magnitudes may influence probabilistic people relatively less and non-probabilistic people relatively more. Given a low purchase requirement, non-probabilistic people construe a “high probability” of reward and non-probabilistic people a “low effort” requirement. In this case, probabilistic people may distinguish between different reward magnitudes, because there is a significant chance the reward will actually be received. On the other hand, non-probabilistic people may not distinguish between reward magnitudes; both small and large rewards may reciprocate low effort. In sum, given a low purchase requirement, the difference in reward magnitudes may influence probabilistic people relatively more and non-probabilistic people relatively less. To investigate these predictions, we used a probability prompting manipulation (this manipulation was previously used for a somewhat different purpose by Erev et al (1993) and Erev and Wallsten (1993)). In the prompted conditions participants first estimated the probability that they would actually eat the required number of sandwiches (were they to join) and then indicated whether they would join the program. Having participants estimate the probability of earning the reward should induce a probabilistic mindset. In the unprompted conditions, participants first indicated whether they would join and only afterwards estimated the probability that they would eat the required number of sandwiches. We expect unprompted participants to adopt a non-probabilistic mindset. Our predictions can be summarized as follows. Given a high purchase requirement (low probability, high effort), reward magnitude will have greater influence on unprompted participants’ tendency to join. Decision Making without Likelihood Judgment 9 Conversely, given a low purchase requirement (high probability, low effort), reward magnitude will have greater influence on prompted participants. Results Participants’ mean estimates of their probability of meeting the purchase requirement were significantly greater given ten rather than twenty sandwiches, 79% versus 44% (p < .001). Estimates did not vary significantly with reward magnitude, probability prompting, or any interactions involving these variables. Participants’ choices, summarized in Table 1, conformed to our predictions. Given a high purchase requirement, the difference in reward magnitude had little impact on probability prompted participants (42% joined given one sandwich versus 46% given two sandwiches) but a significant impact on unprompted participants (34% versus 60%; p=.02 by chi-square). On the other hand, given a low purchase requirement, reward magnitude had significant impact on prompted participants (58% versus 79%; p<.05) but little impact on unprompted participants (54% versus 57%). In sum, unprompted participants display a pattern of behavior that is exactly the opposite of prompted participants. To reiterate, we interpret this finding as indicating that most unprompted participants did not judge the probability of eating the required number of sandwiches; instead, they may have decided whether to join the program on the basis of a reciprocity rule. It appears that the default mindset in Experiment 1 is non-probabilistic. Discussion A natural question that arises is what factors may make either probabilistic or non-probabilistic mindsets the default in a given situation? Decision Making without Likelihood Judgment 10 First, the salience of intuitive rules promotes non-probabilistic behavior. Many settings will not offer readily attractive rules like a reciprocity rationale; those that do not may yield probabilistic behavior. Second, events that people partially control – participants’ own actions determine whether they fulfill their purchase requirement – may yield more non-probabilistic behavior than events that people do not control (cf. March & Shapira, 1987). We suggest that the formation of intentions can crowd out likelihood judgment. Unprompted participants may have (i) assessed the reward offered as sufficiently (or insufficiently) reciprocating the required effort, (ii) thus formed an intention to make (or not make) the necessary purchases, (iii) then proceeded as if having formed an intention, they were certain to follow it, so that (iv) judgments of the likelihood of fulfilling one’s intentions never arose. This form of nonprobabilistic behavior cannot emerge for events people do not control and thus do not have intentions about. Third, uncertain events may be differentially amenable to likelihood judgment. Judgment of some events proceeds spontaneously, perhaps automatically. For instance, people quickly and easily notice the dissimilarity between political activists and the prototypical bank teller and conclude that a political activist is unlikely to be a bank teller (Frederick 2002). In contrast, judgment of other events may not occur spontaneously. For instance, when asked about the likelihood that a novel transmissible disease will be contained in a particular region within a given period of time, many people do not have a ready response. Moreover, to formulate a judgment, people may draw on numerous considerations; the actions taken by various governments and agencies, the history of similar epidemics, research efforts to find effective treatments, international travel patterns, and so forth. Several factors may discourage people from formulating judgments in this way, including perceived lack of relevant knowledge and the difficulty of integrating disparate considerations into a single judgment. Decision Making without Likelihood Judgment 11 Experiment 2 examines events that participants do not control and also examines one factor affecting the degree to which uncertain events are amenable to likelihood judgment. Participants play the role of CEO of a large conglomerate and must decide on the value of a small start-up. The critical uncertainty concerns whether the start-up will successfully launch its product. Within a larger design, we study two groups of participants, history students and MBA students. History students, we find, believe they lack the knowledge necessary to assess the relevant uncertainty, but MBA students believe they possess the necessary knowledge. Thus, history students should tend to be non-probabilistic and MBA’s probabilistic. Experiment 2: Robots Method Participants were 119 University of North Carolina undergraduates enrolled in a history course and 171 University of Chicago MBAs enrolled in a negotiations course. For each group, the study followed a 2 (implied likelihood of success: low vs. high) x 2 (probability reminding: reminded vs. un-reminded) between-subjects design. The likelihood factor referred to the start-up’s probability of success. Participants read materials implying that the start-up had either relatively high or low chances of success. The materials differed in the bracketed text: You are considering acquiring RoMoCo, a start-up that is trying to develop and bring to market a state-of-the-art robot motion controller. [RoMoCo is currently trying to build a working prototype. If a working prototype can be built, RoMoCo will then try to turn it into a mass-producible device that is not too expensive.] [The controller is a variation on existing military projects. A working prototype has been built. RoMoCo must now turn the prototype into a mass-producible device that is not too expensive.] Financial analyses indicate that if the robot motion controller is eventually brought to market, RoMoCo will be worth $225 million, but if the controller is not brought to market, RoMoCo will be worth only $10 million. A manipulation check, conducted once the experiment had concluded, indicated that mean estimates of RoMoCo’s chances of success were indeed greater given high rather than low implied likelihood (57% Decision Making without Likelihood Judgment 12 versus 35% for history students and 40% versus 22% for MBAs; both p’s < .01). Estimates did not vary significantly with probability reminding or interactions involving this variable. Participants indicated the most they would bid to acquire RoMoCo, using a scale starting at $10 million, followed by $25 million, and subsequently demarked in increments of $25 million up to $200 million. Participants also explained their bid in writing. In the present setting there are two ways to distinguish probabilistic and non-probabilistic behavior. First, only probabilistic behavior will vary with RoMoCo’s implied likelihood of success. Under probabilistic mindsets, participants’ bids should be greater in the high than low (implied) likelihood conditions, but under non-probabilistic mindsets, participants’ bids should not vary with likelihood. Second, probabilistic mindsets should engender explanations invoking RoMoCo’s likelihood of success, whereas non-probabilistic mindsets should engender explanations invoking rules concerning other considerations. We operationalized the tendency to adopt probabilistic or non-probabilistic mindsets in two ways. First, to reiterate, we suggest that the default mindset is non-probabilistic for history students but probabilistic for MBA’s. After the experiment, we had new participants rate their “knowledge and understanding of the factors that determine start-ups’ success in bringing new products to market” and their “familiarity with market analyses.” On 10-point scales, mean self-ratings were 2.7 and 2.1 for history students versus 5.2 and 6.9 for MBA’s. Thus, relatively speaking, history students believe they lack expertise in judging start-ups’ chances of success, whereas MBA’s feel they possess such expertise. If such considerations influence default mindsets, history students will tend to be non-probabilistic and MBA’s probabilistic. Second, we included a manipulation intended to facilitate probabilistic thinking. Specifically, participants in the probability reminding conditions read the following paragraph: Decision Making without Likelihood Judgment 13 The financial analyses are not meant to address, and therefore do not provide an estimate of, the probability that RoMoCo will succeed in bringing the controller to market. Thus, you must form your own estimate of RoMoCo’ chances of success. Participants in the un-reminded conditions had these instructions omitted. The present “probability reminder” should be distinguished from Experiment 1’s “probability prompting.” Under probability prompting, participants provided numerical probability estimates. Probability reminding encourages participants to judge RoMoCo’s chances but does not require any overt response. Because MBA’s are expected to be probabilistic by default, the reminder should have no effect on them. However, because history students are expected to be non-probabilistic by default, the reminder should influence them. Our predictions can thus be summarized as follows. All MBA participants and reminded history students will be probabilistic; their bids will vary with RoMoCo’s chances of success and their bid explanations will discuss RoMoCo’s likelihood of success. In contrast, un-reminded history students will be non-probabilistic; their bids will be unaffected by RoMoCo’s chances of success and their explanations will reflect rules concerning what Shapira referred to as “arbitrary factors.” Results and Discussion The pattern of bids, summarized in Table 2, supports our predictions. For MBA’s, mean bids were greater given high rather than low implied likelihood of success, both with and without a probability reminder; $48 versus $28 million (p < .01) and $51 versus $25 million (p < .01), respectively. In contrast, for history undergraduates, the presence of a reminder was crucial. With a reminder, mean bids were greater for high rather than low likelihood, $78 versus $57 million (p < .05); without a reminder, mean bids were about the same in either condition, $71 million versus $72 million. We examined participants’ bid explanations in two ways. First, we sorted explanations according to whether they included the following probabilistic terms: chance(s), odds, percent, probability, risk, expected value, likely, and likelihood. Given a reminder, 50% of history students’ explanations included Decision Making without Likelihood Judgment 14 probabilistic terms, but without a reminder only 26% did so (p<.05 by chi-square). A different pattern arose for MBA’s, 62% used probabilistic terms given a reminder and 60% without one. These data support the assertion that history students are non-probabilistic by default but may be reminded to be probabilistic, whereas MBA students default to probabilistic mindsets. Second, an independent judge, unaware of our manipulations and predictions, coded explanations as probabilistic versus non-probabilistic on a more interpretive basis. The judge was instructed to code as probabilistic any explanations that “implicitly or explicitly estimate or compare the likelihood of RoMoCo’s bringing or not bringing the controller to market” and to code all other explanations as nonprobabilistic. To illustrate, the explanation below was deemed probabilistic: “[I] didn’t want to bid too much, because there’s greater chance of failure than success” The following explanation was coded as non-probabilistic: “I think that if you pay $75 million, and it will be worth $225 million if brought to market, you can spend millions on marketing and other things and still come out very profitable.” Indeed, this latter explanation concerns Shapira’s “arbitrary factors.” Marketing expenditures are an important consideration, but arguing that opportunities for such expenditures exist is different from evaluating the likelihood of RoMoCo’s success. Participants’ explanations again revealed the predicted distinctions. For history students, 35% of explanations were coded as probabilistic with a reminder but only 18% without one (p<.05 by chisquare). For MBA’s, 51% of explanations were coded as probabilistic given a reminder and 45% without one. Decision Making without Likelihood Judgment 15 The finding that un-reminded history students’ bids did not vary with likelihood does not imply that these participants failed to expend the effort necessary to generate sensible responses. On the contrary, it appears that the effort of un-reminded history students may have exceeded that of other participants: un-reminded history students’ explanations were longer than those of reminded history students (32 versus 24 words, p<.05; un-reminded and reminded MBAs’ explanations were shorter still, 22 and 20 words, respectively). Indeed, the explanations proffered by un-reminded history students, such as the passage concerning marketing expenditures, though non-probabilistic, were entirely sensible. Shapira’s observation that executives (many of whom hold MBAs) often fail to judge likelihood – in high-stakes settings – is instructive in this regard. Given the simple setting of Experiment 2, an MBA background may be sufficient to induce a probabilistic mindset. However, in the real world of messier, more complex situations many people may approach many decisions (about start-ups, sandwich programs, travel to China) by focusing on non-probabilistic rather than probabilistic variables.